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Augmented Reality technology may help improve Air Traffic Control Tower efficiency and 
safety during low-visibility conditions. This paper presents the assessments of five off-duty 
controllers who ‘shadow-controlled’ with an augmented reality prototype in their own facility. 
Initial studies indicated unanimous agreement that this technology is potentially beneficial, 
though the prototype used in the study was not adequate for operational use. Some controllers 
agreed that augmented reality technology improved situational awareness, had potential to 
benefit clearance, control, and coordination tasks and duties and could be very useful for 
acquiring aircraft and weather information, particularly aircraft location, heading, and 
identification. The strongest objections to the prototype used in this study were directed at 
aircraft registration errors, unacceptable optical transparency, insufficient display 
performance in sunlight, inadequate representation of the static environment and insufficient 
symbology. 


I. Introduction 

Air traffic control Towers suffer visibility impairments during inclement meteorological conditions that negatively 
impact airport arrival rates, airport departure rates, terminal-area acceptance rates at en route coordination-fixes, traffic 
flow management constraints, and surface vehicle accidents, including runway incursions. 1 ' 2,3,4,5 There is no validated 
method for fully mitigating the operational impact of impaired Tower controller visibility. This technology gap is 
challenging for both current traffic management operations and the Next Generation Air Traffic System (NextGen) 6,7 
requirements for equivalent operational capabilities in all meteorological and visibility conditions. 

Several independent research activities have proposed designs that use head-worn, head-tracked, see-through 
display sub-systems to enhance Tower controllers’ situation awareness. 8 ’ 9 ’ 10,11,12 ’ 13, 14,15 Most of these proposals were 
influenced by previous research in three-dimensional displays developed for display of traffic information to the 
pilot 16,17 , air traffic simulation 18 , and flight deck avionics 19,20,21 . Though this previous work had diverse motivations, 
they had in common a technological approach that required high performance real-time three dimensional graphics 
computation as a critical element of computer-human interface. In most cases, this work was based on experiments 
conducted in indoor laboratories, where most conditions (such as lighting) could be conveniently controlled. Although 
these studies clearly showed that Augmented Reality (AR) offered the potential for improved Tower operations, none 
actually evaluated the concepts in an operational environment using a research prototype built from available 
technologies. 

The purpose of the current study was to move the scene of investigation for AR concept exploration out of the 
laboratory and into the field. The study presented below reports five Tower controllers’ assessments of the potential 
benefit of application of this technology, based on their experience using an augmented reality model prototype during 
‘shadow control’ sessions inside their own operational Tower facilities. This study was the first time AR technology 
was evaluated by a Tower controller team in their own Tower using real-time ATC radar data and observations of live 
air traffic. 

The Augmented Reality Tower Technology (ARTT) research prototype used in this study displayed both dynamic 
real-time air traffic data (e.g., aircraft location, identity, speed, and distance), and computer-generated graphics that 
correlate to the static environment objects (e.g., horizon line, compass rose), superimposed on the controller’s view 
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from the Tower of the actual objects. In a previous publication the authors described the design approach of ARTT 
systems in greater detail, and presented preliminary results of the controller assessments, including responses to 18 of 
the 139 Likert-scale questions that are analyzed below in the present report. 22 This paper presents a more 
comprehensive description, reporting, and analysis of a controller’s evaluations. A subset of 79 questions addresses 
the potential of AR technology to benefit well defined Tower tasks and duties, such as clearances and control, non- 
| control activities, and vehicular and inter-facility coordination. Another subset of 26 , questions address potential 
information acquisition benefits, including complementary functionality with the current operational Tower radar 
display. The research prototype apparatus was evaluated for comfort, aircraft data block display and situation 
awareness utility, and head mounted display and tracking performance. The controllers were also asked to assess the 
potential of augmented reality technology to benefit Tower operations, and the acceptability of the prototype 
implementation used in the field study. 


Ron Reisman 4/23/08 10:56 AM 


Deleted : 5 J 


IT. Background 


A. Low Visibility Tower Operations 

Tower controllers are required by FAA regulations to visually observe relevant vehicles and their environment 
whenever possible. Although a discussion of the Tower controller methodology is beyond the scope of this paper, one 
essential difference between the Tower and the other ATC domains is that only Tower controllers visually observe the 
3-D world, rather than use 2-D plan- view displays as their primary sources of information. Tower staff must also use 
the several “heads-down” 2-D displays and text-based systems found in the Tower cab. Tower controllers integrate 
the diverse information to address a well-defined set of tasks and duties. These Air Traffic Control Tower (ATCT) 
tasks and duties include: issuing ground clearances, approach and over-flight control instructions; addressing runway 
incursion issues; and providing Tower-to-aircraft coordination under regulatory constraints that are dependent on 
visibility and weather conditions. In most airports, most ATCT coordination is with surface vehicles, including taxiing 
aircraft, fuel and baggage, crash fire rescue, grounds keeping and wildlife control. In addition to low (and no) 
visibility conditions induced by weather, it is also possible for the airport, surrounding airspace, and vehicles to be 
obscured by architecture, high-glare conditions, and the cover of night. 

Previous studies have identified large fuel savings and ancillary economic benefits if a system could be 
implemented that would insure stable rates of airport capacity in all visibility conditions. Shackleford and Karpe 
(1996) calculated that a ‘Low-Visibility Tower Tool’ could facilitate 196M Kg. of aviation fuel savings per year, 
based on deployment at 42 airports in 201 5. 23 The studies concluded that a system capable of maintaining the safety 
and efficiency of surface operations during adverse visibility would produce a variety of benefits. They also 
concluded that higher average airport arrival rates would enable more uniform and productive en route and Traffic 
Flow Management operations. The increased surface capacity reliability would improve metrics for taxi-times, 
departure queues, ground-delays, ground-holds, and cancellations. Their theoretical low-visibility Tower display was 
required to provide equivalent situation awareness to a clear 360-degree field of regard, encompassing the airport 
surface and all approaches and departure areas in the Class-D airspace. 
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B. Augmented Reality Technologies 

There has been increasing interest in using AR technology to enable Tower controllers to operate equivalently 
under visual and instrument meteorological conditions. The independent design consensus is that head-worn AR 
systems may potentially fuse relevant data from Tower systems (e.g., air traffic control and surface surveillance radar) 
and display appropriate information as an overlay on the controller’s view of airborne and surface vehicles. 

Most apparatus configurations used for research prototype in ARTT studies may be viewed as five (theoretically 
independent) sub-systems, as illustrated in Jig. 1. These sub-systems are: 3-D visual database, traffic management 
information, head motion and orientation tracker, computer generated graphics, and see-through display. These five 
subsystems are required to present a ‘virtual’ world of digital information that is optically combined with the human 
controller’s view of the ‘real’ world. In some cases, the subsystem implementations are interchangeable. For example, 
several different head-tracking technologies or optical combiner displays may be used as interchangeable modules. 
The paradigmatic configuration includes: see-through head-worn display, head tracking system, and computer- 
generated graphical representation of the surrounding environment and vehicles. 
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Figure 1. Generic Augmented Reality Tower Technology conceptual model. 
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III. ARTT Research Prototype 

The apparatus used for this study is illustrated in F ig . 2. The photograph shows a shadow-controller jn the Moffett^ 
Towe r cab, located at Moffett Field, California. Th ^ off-duty controllers,, wor^, fee Sony Glasstron head .worn display. \ 
-with .an Intersense InertiaCube head-tracker mounted on the clear plexiglass post (visible directly above the 
controller’s head) . The head-tracker enable ' determination of observer’s field-of-view. The two laptops used i n this 


research prototype are visible to the left of the controller. The first laptop r n NASA’s Center TRACON Automation 
System (CTAS) software lo acquire, real-time Northern California TRACON (NCT) system and surveillance data. 
The second laptop used, this NCT d at a as input to the ARTT software that, created the 3-D perspective view that is 
displayed to the controller. The CTAS software ran under the Linux operating system, and the ARTT software ran 
under Microsoft XP. The subsystems used in the ARTT experimental prototype are described in greater detail below T 
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A. Computer SoftwareAHardware 

The graphic display system used with ARTT was based on re-used code originally created for use at the NASA 
Ames Research Center Aerospace Human Factors Laboratory. This code was principally written by Michael Hill 
under the direction of Stephen Ellis and Bernard Adelstien. The software was written in C++, using the GLUT 
| OpenGL open source libraries, and was used in a variety of human factors studies ^ The code was modified to read 
real-time TRACON flight plan and aircraft state data created by CTAS. 

This software was used in the ARTT research prototype for the field tests described below. In this study the ARTT 
software ran on a Dell Laptop configured with an Intel 1 .2 GHz Pentium and 1 GB RAM. This system maintained a 
minimum graphics frame update rate of 30Hz, and was observed to run as fast as 55 Hz. 
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Figure 2. Augmented Reality Tower Technology research prototype used for assessments by shadow- 
controllers at Moffett Tower. 


B. Prototype Display Symbology 

| Jig. 3L illustrates the symbology used in the study. Computer generated graphical red circles with 2-degree 
diameters identified the computed position of each aircraft, based on aircraft surveillance report real-time data. A 
three-line data block was associated with each of these aircraft marker circles. The first line of the data block 
contained the aircraft call-sign. The second line contained the horizontal distance (in nautical miles) of the aircraft 
from the viewer’s position in the Tower. This value was derived from evaluating real-time position data reported by 
the FAA TRACON data, in relation to the position of the viewer. The third line displayed the aircraft’s altitude in feet. 

Ground reference was represented by a computer generated green horizon line, with compass heading markers at 10 
degree intervals. Each interval had a vertical line with the compass heading displayed above it. The orientation of the 
| viewer’s head was also displayed, expressed as , magnetic heading (displayed at the top of the frame), pitch (displayed 
on the right), and roll (displayed at the bottom of the frame). 
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C. Optical Combiner (‘See-Though’) Display 

The ARTT research prototype used a Sony Glasstron PLM-S70C t head-worn display. This particular display was 
unanimously assessed to be unacceptable for operational use in a variety of lighting conditions. This is not particularly 
surprising, considering that the Glasstron was originally designed as a consumer electronics ‘personal theater’ product, 
and was not intended for see-through display applications. The ‘transparency’ of the this display is comparable to dark 
sunglasses, with the manufacturer specifying 20% transmissivity Consequently it is not optimized for “see-through” 
performance. In spite of these known limitations the Glasstron has been used in many AR studies. 26, 27 
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Figure 3. Symbology used in ARTT research prototype display. The callout labels (in white) are illustrative, 
and are not shown in the actual prototype. The areas that are black in the above illustration are transparent 
when viewed through a see-through head mounted display. 


D. Head Motion and Orientation Tracker 

The ARTT prototype used an InterSense InertiaCube2 motion tracking hardware 28 and AuSim AST manager 
software 29 to determine head orientation data required for viewport and viewpoint calculations. The InertiaCube2 
incorporate ^ an accelerometer, an angular rate gyro, and magnetic compass. The AuSim AST software interpreted 
Jracker data, app liedf a ppropriate filters, and servpd jfae derived orientation data through shared-memory interprocess 
communication. The ARTT application software accesse d head-orientation information via this shared-memory 
interface T 

The Inertiacube2 was selected because it uses inertial tracking, and measures absolute orientation relative to the 
earth’s gravitational and magnetic fields, and therefore does not require a separate external reference. This is 
convenient for use in areas where it may be inconvenient to erect an external position reference, such as operational 
ATC facility. Unfortunately this technology is susceptible to interference from environmental metal and 
electromagnetic interference that create intermittent artifacts, and even under optimal conditions routinely exhibits 
average orientation errors of two degrees. These limitations and artifacts of the head tracking system are well- 
understood as a significant source of static and dynamic registration errors 30, 31 ' 32 ' 33, 34 
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E. Traffic Management Information 

The accuracy of the ARTT representation of vehicular position is dependent on the quality of the real-time traffic 
information, particularly positional information provided by air traffic control surveillance systems. The current 
operational FAA traffic management system used for Terminal area operations is the Automated Radar Tracking 
System (ARTS). The ARTS positional state data is based on Airport Surveillance Radar, Model 9 (ASR-9), which has 
a 4.8 second sweep update rate. In addition to surveillance state data (e.g. aircraft position, vertical and horizontal 
speed), the ARTS data contains FAA ATC systems data, such as filed flight information (e.g. aircraft type and 
equipage) and inter- facility messages. 
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F. Prototype Limitations 

The prototype used in this study had known limitations and was not intended to meet operational requirements. 
The prototype was primarily used to communicate the characteristics of AR technology to the controllers so that they 
may imagine how a more capable instantiation may be beneficial for Tower operations. Each of the prototype’s 
subsystems’ known deficiencies contributed to the prototype’s performance issues. 

For instance, the InterSense head-tracking orientation sensor’s specified 2-degree orientation accuracy contributes 
to a potential 2-degree error in overlaying aircraft position symbols over the actual optical image of the aircraft. The 
InertiaCube’s magnetic compass artifacts in this sensor and the resultant excursions would not be acceptable for 
operational equipment. 


The FAA NCT supplied the real-time terminal area data used in these studies. This NCT .real-time data 


infrastructure was originally designed for the FAA ARTCC Traffic Management Advisor (TMA) operational system 
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requirements. The FAA has made this data available to NASA for use in research activities. The Likert test results 
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presented below relied on this current-technology NCT realtime surveillance data. One limitation of the study 



presented below is the reliance on the current surveillance technology, rather than systems that more realistically 
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model the data quality expected in the NextGen environment. The NCT data’s relatively slow update rates (almost 5 
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seconds) further contributes to registration errors. The Glasstron display’s low transmissivity and insufficient display 
brightness make it unsuitable for use in the Tower’s full range of lighting conditions. 


IV. Assessments and Evaluations 


A. ARTT Field Study Design 

The field study presented below were performed using the research prototype discussed in the previous section by 
five Moffett Field ATCT controllers at the Moffett Tower. Moffett Tower was originally built by the U.S. Navy and is 
currently operated by the Department of Defense as a contract Tower for NASA. The Moffett Tower is an operational 
Tower, albeit one that only handles a fraction of the workload it carried when Moffett Field was a Naval Air Station. 
As a result the Moffett Tower currently often has periods of relatively low activity when research may be prudently 
pursued without impacting operations. During such periods, non-interference concept exploration studies may be 
undertaken from the Tower cab, subject to Tower Chief approval. One form of study involves the participation of off- 
duty Moffett Tower controllers who assess ARTT research prototype and technology concepts while on-duty 
controllers concurrently perform actual operations. The ARTT protocols required off-duty controllers who engaged in 
research to maintain a safe separation distance (~6 ft.) from the on-duty controller, and to maintain a non-interference 
| posture at all times, as illustrated in 4. 
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Figure 4. Field study floor-plan at NASA Ames Research Center’s Moffett Field Air Traffic 
Control Tower. The off-duty controller subjects may ‘shadow-control’, while the on-duty 
controllers concurrently perform the operational tasks. 
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In the study described below, the ARTT research prototype was not used to actually control an aircraft. Off-duty 
controllers assessed the augmented reality prototype in their Tower while on-duty controllers performed normal 
operations. This type of study is generically referred to as ‘shadow-controlling.’ The controllers who participated in 
this study had an average of 13.6 years ATCT experience, with standard deviation of 6 years. They had worked for an 
average of 4.6 years at Moffett Tower, with standard deviation of 5.5 years amongst the cadre. None of the controllers 
had any experience with AR technology prior to their exposure during the ARTT assessment studies. 22 

The ARTT engineering prototype was designed to illustrate AR concepts and also identify gaps in practical 
implementation. The controllers filled out a series of Likert-scale questions after each session as part of the assessment 
documentation. The range of the Likert scale was five units, representing with ratings of (in descending order): 
‘Very Useful’ (5); ‘Useful’ (4); ‘Neutral’ (3); ‘Not Very Useful’ (2); and ‘Useless’ (1). The Likert scale questions 
were divided into five groupings, described below. 

The controllers’ Likert-scale responses are presented below in ‘Box and Whiskers’ plot format, also known as 
‘Tukey Plots.’ This is a standardized way of displaying the distribution of data based on the five number summary: 
minimum, first quartile, median, third quartile, and maximum. In these plots the bottom of the boxes represent the 
values for the first quartile, the median is the value for the second quartile and is plotted as horizontal bar within each 
box, and the third quartile value is represented by the top side of the box. The central rectangular box spans the first 
quartile to the third quartile (i.e. the interquartile range). The ‘whiskers’ are the lines that typically extend from the 
boxes, representing the maximum and minimum value by the respective locations of the top and bottom whiskers. 35 


B. Tasks and Duties Applicability Assessment 

The first set of 79 questions addressed the potential of augmented reality technology to assist controllers in+ 
performing their primary duties. All of the questions in this set were prefaced with: “How useful would ARTT 
technology be in performing the following tasks:” These questions are grouped into six categories described below: 
‘Tower Clearances and Control,’ ‘Inter-Facility Coordination,’ ‘Air Traffic Coordination,’ ‘Surface Traffic 
Coordination,’ ‘Primary Duties,’ and ‘Tasks (excluding control).’ The context of these questions is therefore the 
potential benefit of AR, not the actual performance of the particular ARTT research prototype used in the trials. The 
questions then listed a particular task that each controller rated with the five-level Likert-scale described above. The 
‘Tasks and Duties’ assessment questions were grouped into six categories: 

1. Tower Clearances and Control 

These 9 questions covered controllers’ tasks associated with issuing instructions and clearances to aircraft. 
Specifically, the controllers were asked to assess the potential usefulness of AR technology for the following duties: 
ground control, local control, issuing ground clearances, overflight control instructions, arrival control instructions, 
departure control instructions, flight data processing, clearance delivery, and issuing Instrument Flight Rule (IFR) 
clearances. 


2. Inter-Facility Coordination 

This set of 8 questions addressed the potential of AR technology to aid coordination between the Tower and other 
facilities. These questions referred to both the actual facilities that currently coordinate with the Moffett Tower (e.g. 
the Palo Alto, San Carlos, and San Jose Towers) and the generalized utility of this technology for genera of 
coordination (e.g. generalized Tower-to-Tower coordination). Specifically, the controllers were questioned about: 
Moffett to Palo Alto Tower Coordination, Moffett to San Jose Tower Coordination, Tower-to-Tower Coordination 
(generalized), Moffett to NCT Coordination, Tower-to-TRACON Coordination (generalized), Moffett to San Carlos 
Tower Coordination, Moffett to San Francisco Tower Coordination, and Moffett to Oakland Coordination 

3. Air Traffic Coordination 

This set of 17 questions addressed tasks and duties associated with Tower coordination with aircraft under a variety 
of visibility conditions. In addition to the normal types of vehicles that would be encountered at most Towers, there 
were also several questions regarding more specialized vehicular traffic that is controlled by Moffett Tower (e.g. 
bundle-drops, parachutists, un-crewed aviation vehicles aka UAV). Specifically, the controllers were questioned about 
| coordination with air traffic obscured by architecture or weath e nuring high-glare and IFR conditions; in no-visibility 
conditions; and in low visibility conditions. Other coordination tasks included: Tower to fixed-wing aircraft 
coordination. Tower to rotorcraft coordination. Tower to general aviation coordination. Coordination with Air Traffic 
at night; Coordination with Air Traffic in daylight; Tower coordination with Air Traffic (in general); Coordination 
with Air Traffic that are clearly visible; Coordination with Air Traffic during VFR Conditions; Tower to bundle-drop 
| coordination; Tower to parachutist coordination, and Tower to UAV coordination 

4. Surface Traffic Coordination 

This set of 21 questions addressed surface traffic coordination issues. Specifically, the controllers were questioned 
about coordination with ground vehicles obscured by weather, in no-visibility conditions, obscured by architecture, in 


Ron Reisman 4/23/08 2:07 PM 


Formatted: Indent: First line: 0" 


Ron Reisman 4/23/08 2:08 PM 


Deleted: 


Ron Reisman 4/23/08 10:17 AM 


Deleted: 


Ron Reisman 4/23/08 10:17 AM 


Deleted: 


7 






low visibility conditions, during high-glare conditions, during IFR conditions, during VFR conditions, in daylight, at 
night, and vehicles that are clearly visible. The were also asked ‘Tower coordination’ questions regarding: Crash Fire 
Rescue Vehicles, Supervisory Vehicles, Security Vehicles, Fuel Trucks, Runway Sweepers, Electricians and other 
maintenance staff, Research Project Vehicles, Tow Vehicles, Mowers and with Ground Vehicles in general. They 
were also questioned about the usefulness of ARTT for addressing runway incursion issues. 

5. Primary Duties 

This set of 12 questions addressed the primary duties encountered at Moffett Tower. These questions referred to 
the specific common configurations for the Moffett airspace. Specifically, the controllers were questioned about 
ATCT duties during West Flow, South East Flow, Memoranda of Agreement (MOA) Amphitheater Extension, MOA 
Hangar Extension, Left-Hand Pattern Traffic, Right-Hand Pattern Traffic, Designated Border operation, and 40x80 
Wind Tunnel operation. The controllers were also asked about the potential usefulness of the technology for cutting 
ATIS and hourly Weather Observation reports, Watch Supervision, and generalized ATCT duties and tasks. 

6. Non-Control Tasks 

The last contains 12 questions which address the non-control aspects of Tower ‘Tasks and Duties.’ The 

questions concerned systems that are used operationally at Moffett Tower for non-control applications. Specifically, 
the controllers were questioned about: operations with the Flight Data Information Operation (FDIO), Remote ARTS 
Color Display (RACD), and Automatic Terminal Information Service (ATIS); evaluating Runway Visual Range 
(RVR), track proficiency time; evaluating and editing Automated Surface Observing System (ASOS) reports with the 
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| Figure 5. Categorized summary of five controller’s responses to 79 ‘Tasks & Duties’ jjkert-scale responses to: 


‘How useful would ARTT technology be in performing the following tasks?’ 
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The context of these questions is therefore the potential benefit of AR, not the actual performance of the particular 
ARTT research prototype used in the trials. The medians indicate that they found ARTT technology clearly useful for 



four of these six categories, and neutral for ‘Inter-Facility Coordination’ and ‘Tasks (excluding control)’ categories. 
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In addition to the 79 ‘Tasks and Duties’ questions described above, there was a final subset of 5 additional ^ 
questions that addressed the issue of technology preconditions required to use AR technology for Tower applications. 
These five questions had the form: ‘AR technology would be useful in performing the ATCT tasks only if:' system- 
fusion is improved or computer ‘augmented' display is improved or see-through visibility is improved, or certain 
features are developed, or procedures are changed. These results are illustrated in Fig. 6, and are consistent with the 
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Figure 6. Controller’s responses to the five ‘only iF questions indicate their assessment that AR technology 
for Tower applications will require more feature development, improved see-through visibility, and better 
display performance. They strongly agreed that system and information fusion must be improved. 


C. Information Acquisition Technology Assessment 

A set of 14 questions addressed the issues of the potential of AR technology to aid in the acquisition of 
information. These questions had the form: “How useful would ARTT technology be in acquiring the following 
information?” This set was conceptually divided into two categories, described below. 

1. Functional Comparison with Operational Radar Display 

A set of six questions compared the potential utility of AR technology with current Tower operational radar system 
functions. Controllers were asked to assess the capability of ARTT relative to the performance of their primary radar 
system, the Remote ARTS Color Display (RACD). The ARTT prototype itself did not have display options 
comparable to the RACD. The controllers were asked to imagine how AR technology could be adapted and deliver 
data currently provided by the RACD. 

In certain specialized cases, such as ‘Preview Areas,’ ‘Tab Lists,’ and ‘P-Lists,’ the potential advantages of ARTT 
were evaluated as ‘Neutral’ in comparison to their current system’s functionality. The controllers’ opinions were 
particularly strong regarding the potential of ARTT to become more useful than the current RACD plan-view display 
for ‘Aircraft Track and Target Display Options.’ Four out of the five controllers responded with the highest rating, 
‘Very Useful,’ and only one disagreed, finding ARTT ‘Not Very Useful’ for tracking aircraft in comparison with the 
standard radar display. Though the ARTT prototype could not display the RACD ‘Map,’ ‘Weather,’ ‘Tab List,’ and 
‘P-list’ Display Options, the controllers imagined that using AR technology would add benefit if it could be 
appropriately modified to display this information. The results for each of the questions associated with the RACD are 



summarized in Jig. 7. 
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| figure 7. Controllers’ assessments of ARTT’s potential utility for several display functions. Each of these 


categories is related to a function of the Remote ARTS Color Display (RACD) System, the primary radar 
display currently used in Towers. 
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2. Information Acquisition 

A set of 8 questions addressed the potential of AR technology for improved acquisition of specific information and 
situational awareness. The controllers’ responses were consistent with their positive response to its potential advantage 
over the current radar system for tracking aircraft, .fig. 8. summarizes the controllers’ responses. In six of the eight 


questions the controllers consistently (though not unanimously) responded with ‘Very Useful’ in acquiring information 
regarding aircraft location, heading, data-block details, and surface vehicle location and situation awareness. The 
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remaining two questions concerned air traffic and Tower situation awareness, the controllers responded that ARTT 
could be ‘Useful.’ The controllers generally gave enthusiastic ‘Useful’ or ‘Very Useful’ responses to most questions 
regarding the potential of AR technology to assist in ATCT tasks, duties, particularly regarding information 
acquisition. The controllers’ responses consistently assessed the technology as potentially very useful (though there 
were a few outliers). There was less agreement about the usefulness for aircraft situation awareness than in other 
categories. 
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Figure 8. Information Acquisition Question: How useful would ARTT technology be in 
acquiring the following information? 
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D. Research Prototype Comfort 

The head-mounted display used in the field trials was not acceptably comfortable. The average response to the 
Likert-scale question “Please rate how comfortable you feel working with the ARTT prototype” was 2.75 (std dev: 
0.96), where a 2 represented “not comfortable” and a 3 represented “comfortable.” This was consistent with related 
questions that were included in the evaluation of the research prototype, in which the average head mounted display 
comfort rating was 2.4 (std dev: 0.89). 

The vendor specifies that the Glasstron has an optical transmissivity of 20%, comparable to dark sunglasses, 
making it unusable during low light conditions. For instance, the controllers were not able to see aircraft lights at 
night. It is therefore not surprising that the controllers unanimously agreed that the display was unacceptably difficult 
to use. The controller reactions to this unacceptability evoked a number of spontaneous comments, and they 
unanimously agreed that an AR display for tower applications required greater optical transparency, and that symbols 
and text should be readable in all lighting conditions and at all times. 

The current head-mounted digital video display technologies, however, constitute a ‘vision impairment’ as 
evaluated by current FAA guidelines. The controllers in this study generally consider any see-through displays with 
less than 50% transmissivity as an unacceptable optical impairment. Several of the controllers complained of visual 
strain for at least part of the 45 minute trials, and two reported momentary feelings of nausea that they attributed to the 
disorienting effects of the apparatus. 


E. Research Prototype Performance 


trials. The controllers were instructed to assess the specific prototype used in the trials, rather than assess the potential 
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| This set of jj, questions addressed the acceptability of the data block information presented by the AR research 
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This set of j_ U questions addressed the performance of the head mounted display (HMD). Specifically the 
controllers were asked about the acceptability of the HMD’s visual transmissivity, the visibility of ‘real-world’ seen 
through the HMD , the brightness and clarity of HMD computer-generated text and images, and HMD performance 
during various lighting conditions, such as typical daylight, night-time, and high-glare situations. There were also 
summary questions regarding the overall acceptability of the HMD performance and comfort. 

3. Situation Awareness 

This set of 4 questions addressed situation awareness issues. Specifically the controllers were asked about surface 
vehicles, specific aircraft, air traffic (multiple aircraft) and overall situation awareness. 

4. Head-Tracking and Latency Performance 

Three questions addressed the controllers’ assessment of head tracking and system latency performance. Many of 
these perceived latencies apparently due to the TRACON surveillance radar characteristics, such as the 4.8 second 
update rate. The controllers are not experts in this technology, and their responses were not sought as part of 
engineering diagnosis and analysis. Instead, they were asked for their perceptions of the engineering prototype’s 
performance and their opinions on the nature of shortcomings. The specific questions addressed perceived latencies 
apparently due to head-tracking & display processing, and perceived accuracy of head -tracking sensor systems. 

The results from these four categories are summarized in Fig. 9. The number of questions in each category is 
noted in each interquartile box fas in Fig. 51. The exception to this notation convention is ‘Head-Tracking & Latency 
Performance’ since there is no visible interquartile box due to the consistent ‘Unacceptable’ responses by most 
controllers to all three questions in this category. 

The controllers’ consensus was that the data block information did not provide any particular benefit over the data 
blocks currently available from the RACD. and the head-mounted display performance was unacceptable, the overall 
effect on situation awareness was unacceptable, and the head tracking and perceived latency of the AR system was 
most unacceptable of any category. Post-session interviews clarified that the controllers’ complaints regarding 
‘latency’ appeared to be related to the relatively slow (~5 second! update rate of current Terminal -area surveillance 
radar systems. They were also dissatisfied with the angular errors attributable to head-orientation tracking anomalies. 
The controllers were also generally dissatisfied with the apparatus comfort. The general comments were that an 
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acceptable head mounted display should be no more invasive than standard glasses, with at least 50% optical 
transmissivity, and with display illumination sufficient to be effective throughout the full range of lighting conditions. 



Figure 9. Categorized summary of controllers’ responses to 2 & questions regarding the performance of 
the ARTT research prototype. 

v 

F. Controller Summations 

This set of 12 questions addressed the controllers’ summary evaluations of the specific ARTT research prototype 
used in the study, and the potential of a mature AR technology to benefit Tower operations. These questions had the 
form of: “Please rate your agreement to the following statements.” These 12 questions were divided into two 
categories, described below. 

1. ARTT Research Prototype 

This set of 6 questions addressed the controllers’ evaluations of the research prototype. Specifically, the 
statements proposed that the ARTT prototype computer-generated display symbology must be improved, and 
(conversely) that the symbology is useful; that the displays are perceived adequately; that (conversely) the HMD 
impairs real-world vision, and that the prototype (as -is) could be useful in an Tower, or that a slightly improved (as 
opposed to ‘completely re-engineered’) version of the prototype could be useful. The controllers’ responses are 
summarized in 1 0. 

The controllers agreed that the prototype HMD constituted a ‘vision impairment’ rather than a beneficial 


‘augmentation.’ They also agreed that this particular research prototype did not generate acceptable displays, required 
improved symbology, and generally required extensive improvements before it would be useful for Tower operations. 

In post-trial interviews the controllers unanimously voiced concerns about the accuracy and correspondence of the 
computer-generated information with the actual views from the Tower. The prototype display’s static reference was a 
compass-rose horizon line with 10 degree interval markers. Controllers typically disliked this compass rose display, 
and unanimously requested more three-dimensional (3-D) representations of visual cues. In post-session interviews 
they often commented that they do not normally visualize the world as ‘points on a compass.’ Instead they often used 
visual landmarks for orientation. During the interviews all mentioned that they would have liked to have seen more 
representation of static features, such as runways, taxiways, non-moving objects, architecture, terrain, and similar 
landmarks. 

The results of these Likert tests were reinforced bv post-trial interviews and bv a standard Controller Acceptance 
Rating Scale (CARS') questionnaire. 36 The controllers unanimously responded negatively to the first CARS decision 
point “Is the system safe and comfortable?” and positively to a resultant mandate for improvement. 
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Figure 10. Controllers’ assessments of the research prototype used in the ARTT field studies. 
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2. AR Technology Potential 

This set of 6 questions addressed the controllers’ assessment of the potential of a future ‘perfected’ AR technology 
to benefit Tower operation. Specifically, the statements proposed that ARTT Technology could be useful for ATCT 
tasks and systems, or (conversely) that it will never be useful for ATCT tasks and systems; that it requires new aircraft 
data block information and/or formats, and/or new decision-support-tool operational concepts. The results are 
illustrated in Jig, 1 1. 
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Figure 11. Controllers’ summary assessments of AR potential for Tower applications. 

The controllers generally agreed with each other more consistently than with most of the other questions. The most* 
consistency was exhibited in the final question in the series: ‘I believe it is worthwhile to support ARTT research & 
development.’ This was the only question (out of 139) in which the controllers unanimously provided the same 
answer: ‘Strongly Agree. In post-trial interviews the controllers often expanded on this last question bv advocating 
further development and expressing their acceptance that AR technology may become very useful for NextGen Tower 
facilities^ 
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V, Concluding Remarks 


The prototype, though insufficient for operational use, was sufficient for illustrating the potential of AR 
technology. The controllers strongly agreed on several key areas of the AR technology assessments. Their principal 
findings were: AR technology is potentially useful for primary ATC Tower tasks and duties. The technology may be 
particularly useful for clearance and control tasks, and may improve coordination with air and surface vehicles. The 
controllers strongly agreed that AR technology could complement their current primary surveillance radar systems, 
particularly for aircraft track and target displays, map display options, and weather displays. They were even more 
consistent in their agreement on AR’s potential to improve information acquisition of aircraft location, heading, 
identification, surface location, and awareness of vehicle and traffic situation. Even though there were variations in 
the controllers’ assessments of ARTT’s potential benefit for these tasks and duties, the controllers were very consistent 
in their positive evaluation of the over-all potential of AR technology to benefit Tower operations. 

The major concerns the controllers dealt with the research prototype used in the study. The controllers’ 
consistently complained that the Glasstron HMD was uncomfortable, impractical, and constituted an unacceptable 
visual impairment. The 2-degree nominal errors and magnetic compass excursions contributed to their lack of 
confidence in the system. Many of the controllers’ complaints about the research prototype may be attributable to 
surveillance uncertainties due to the comparatively long update rates (4.8 sec.) characteristic of the TRACON radar 
systems. 

Based on this study, the major challenge in bringing this technology to fruition will be the successful resolution of 
these problems and limitations. Improvements in surveillance , see-through display, and head tracking technologies 
should , be explored _Jo address these engineering problems. It is clear that the controllers were enthusiastically 
supportive of developing AR applications for Tower applications, and that ARTT mav provide benefit in low-visibiltv 
conditions once the technology gaps have been adequately addressed. 
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